Tools

Accuracy Assessment Tools

Accuracy assessment methods can produce statistical outputs to check the quality of a classification result regardless of the method used to classify the image objects.


Accuracy Assessment dialog box
  1. Choose Tools > Accuracy Assessment on the menu bar to open the Accuracy Assessment dialog box.
  2. In the Confusion Matrix drop-down list, select one of the following methods for accuracy assessment:

    • samples - select if you want to calculate and visualize the confusion matrix based on sample objects within a single project.

    • file (flattened) - select if you want to calculate and visualize the confusion matrix across multiple projects within the workspace based on the exported flattened confusion matrix results.

    • object variable - select if you want to calculate and visualize the confusion matrix based on objects defined as samples via an object variable within a single project.

  3. A project can contain different classifications on different image object levels. Specify the Image object level of interest by using the level drop-down menu.
  4. A project can contain ground truth information stored in Image object variables. For the selection of Confusion Matrix based on object variable select the corresponding variable name in this drop-down menu. This option is only activated for selection of Confusion Matrix based on object variable.
  5. Classes - To select classes for assessment, click the Select classes button and make a new selection in the Select Classes for Statistic dialog box. By default all available classes are selected. You can deselect classes through a double-click in the right frame.
  6. Load file (flattened)- This option is only activated for the selection of Confusion Matrix based on file (flattened) and lets you import and visualize an existing flattened file. This file can be created using the export algorithm Export Confusion Matrix .
  7. To view the accuracy assessment results, click Show statistics. To export the statistical output, click Save statistics. Enter a file name in the File name field and save the table in *.csv, *.txt or *.tcsv file format. The statistics file will be saved in the workspace root folder by default.

Confusion Matrix

The Confusion Matrix dialog box displays different statistic types used for accuracy assessment.

Test areas are used as a reference to check classification quality by comparing the classification with reference values (called ground truth in geographic and satellite imaging) based on pixels.

 


Output of the Confusion Matrix based on samples

This statistic type considers samples (not pixels) derived from sample inputs. The match between the sample objects and the classification is expressed in terms of parts of class samples.

The Confusion Matrix dialog box displays the following statistics used for accuracy assessment:

User Class / Sample

The vertical column shows the user classes. The rows in horizontal direction reflect the sample/reference data.

Sum

The Sum row shows the number of objects/pixels that should have been identified as a given class, according to the sample/reference data.

Producer

The producer's accuracy is a false negative, where objects/pixels of a known class are classified as something other than the reference class.

User

The user's accuracy calculates false positives, where objects/pixels are incorrectly classified as a known class.

Hellden

The Hellden index is a mean accuracy index and denotes the probability of a randomly chosen point of a user class and its correspondence of the same class in the same position in the sample/reference data.

Short

The Short’s mean accuracy calculates the ratio of the estimated and sample/ ground truth classes intersection to their union.

Kappa per class

The Cohen's Kappa coefficient (see below) identifies the agreement that is expected when classes are totally independent while Kappa per class calculates the same agreement at per-classes level.

Overall accuracy

The overall accuracy calculates the proportion of pixels out of the reference sites/ground truth sites that were mapped correctly. The overall accuracy is usually expressed in percent, with 100% accuracy being a perfect classification - all reference objects were classified correctly. Range [0,1].

Cohen's Kappa

The Kappa coefficient measures the agreement between two sets - a classification result and ground truth objects/samples, while correcting for agreement that occurs by chance. This statistic is useful for measuring the predictive accuracy in a classification matrix. The Kappa coefficient can range from [-1 , 1], where a value for K = 0 is obtained when the agreement equals chance agreement and the upper limit of Kappa (+ 1.00) occurs only when there is perfect agreement between the two sets.

 

Related to:

Projection and Reprojection

Assign Projection

If the coordinate system of a dataset is unknown or incorrect you can use this tool to assign a projection Tools> Assign Projection, under the precondition that you know the projection information.

Existing information on the coordinate system stored in the dataset will be overwritten. This tool does not reproject data and or perform any calculations. To reproject data open the dialog Tools > Reproject Data.

Supported data formats are tif for image data, shapefile for vector data and point clouds in LAS and LAZ formats.

Assign Projection tool dialog

 

First step is to add the data to the dialog by clicking the Load button.

Load button

Following information is displayed for each file:

To remove a data file from the list, select the file and press the Remove button. To remove all files from the list, press the Remove All button. These two functionalities do not delete the file, the file will be just unloaded from the list of this Assign Projection dialog.

Remove and Remove All buttons

The target coordinate system (Target CS) can be selected or searched from the drop-down list . To search for a coordinate system, start typing a name or an EPSG code of a coordinate system in the field. Alternatively, a target coordinate system can be loaded via file using the From File button to the right of the drop-down.

Defining a target coordinate system

The target coordinate system must be a projected coordinate system. Supported input formats are tif, shp, LAS/LAZ (see also Reprojection tool in following chapter).

In the Output folder field, the path for files with assigned projection is displayed: by default they are located in a subfolder of the data folder called Processed. (If the data is located in different folders, the default path is the folder of the first loaded dataset). However, you can define the ouput folder using the Browse button or type directly into the Output folder field.

Output path for files with assigned projection

 

To assign the new projection to the loaded input files, press the Execute button. Then a progress bar appears on the screen indicating the status of processing. When the projection assignment is done an output path appears next to each file in the dialog.

Reproject

For image analysis in eCognition Developer, the data used must belong to the same projected coordinate system. Geographic coordinate systems have different pixel size in X and Y axes resulting in non-square pixels which should be avoided.

This reprojection tool can be used for reprojecting data from a geographic coordinate system to a projected coordinate system or from one projected coordinate system to another if the data are defined on different systems or if you need the results in a coordinate system different from the one of the input data.

To reproject data select Tools > Reproject Data.

Supported data formats are tif for image data and shapefile format for vector data.

Use the Load button to add the data to the Reproject data dialog.

Load button

In the dialog you get the following information for each file:

To remove a data file from the list, select the file and press the Remove button. To remove all files from the list, press the Remove All button. These two functionalities do not delete the file, the file will be just unloaded from the list of the Reproject dialog.

Remove and Remove All buttons

The target coordinate system (Target CS) can be selected or searched from the drop-down list. To search for a coordinate system, start typing a name or an EPSG code of a coordinate system in the field. Alternatively, a target coordinate system can be loaded via file using the From File button to the right of the drop-down.

The target coordinate system must be a projected coordinate system. Supported data formats are TIF for image data and shapefile for the vector data.

Defining a target coordinate system

 

In the Output folder field, the path for reprojected files is displayed: by default they are located in a subfolder of the data folder called Reprojected. (If the data is located in different folders, the default path is the folder of the first loaded dataset). However, you can define the ouput folder using the Browse button or type directly into the Output folder field.

Output path to the reprojected files

To reproject the loaded input files press the Execute button.

Execute reprojection or close the dialog

Then a progress bar appears indicating the status of processing. Once the reprojection is applied an output path appears next to each file in the dialog window.

Progress bar

If you add image layers or thematic layers to a project, that are assigned to a geographic coordinate system, a warning message is prompted that it is recommend to reproject the data. Select one of the following options: